Skip to content

Update GitHub API integration and versioning; standardize meta keys#100

Merged
goldlabelapps merged 5 commits intomasterfrom
staging
Apr 27, 2026
Merged

Update GitHub API integration and versioning; standardize meta keys#100
goldlabelapps merged 5 commits intomasterfrom
staging

Conversation

@goldlabelapps
Copy link
Copy Markdown
Owner

This pull request introduces a new GitHub integration feature to the application, including API routes, database schema, and documentation to support ingesting and storing GitHub data. It adds endpoints for creating and querying GitHub-related tables in Postgres, provides a table seeding script with realistic test data, and updates the main API router to include these new routes. Minor fixes and documentation improvements are also included.

GitHub API Integration:

  • Added a new FastAPI router (github_router) with endpoints for GitHub data ingestion and querying, including /github for retrieving data and /api/github/createtable for creating tables. (app/api/github/__init__.py [1] app/api/github/github.py [2] app/api/github/sql/create_tables.py [3]
  • Integrated the new github_router into the main API router so GitHub endpoints are accessible from the app. (app/api/routes.py [1] [2]

Database Schema and Utilities:

  • Implemented SQL schema creation for five normalized and flexible GitHub tables: github_accounts, github_repos, github_gists, github_projects, and github_resources, with indexes for efficient search/filtering. (app/api/github/sql/create_tables.py app/api/github/sql/create_tables.pyR1-R156)
  • Added a seeding script to populate GitHub tables with realistic fake data for local development and testing. (app/api/github/sql/seed.py app/api/github/sql/seed.pyR1-R332)

Documentation:

  • Added a comprehensive README for the GitHub API, describing endpoints, table design, and rationale for the schema. (app/api/github/README.md app/api/github/README.mdR1-R38)

Other Fixes and Minor Changes:

Update package version to 3.0.2 and change the CSV import path in the LinkedIn queue endpoint to reference linkedin.csv instead of linkedin_sample.csv. This switches the endpoint to use the actual data file (and preserves the 404 check) likely preparing the app for a new release or production data import.
Update package version from 3.0.2 to 3.0.3. Add app/api/github/README.md documenting the planned GitHub route that will fetch data via the GitHub API and store it in Postgres (database tables not created yet).
Normalize API meta structure: root() now returns "base" (not "base_url") and no longer emits a redundant timestamp; removed the unused epoch variable. make_meta() signature changed from title->message and its returned dict uses "message" and includes "time", "base", "version", and "severity" to provide a consistent meta payload for responses.
Introduce a new GitHub API endpoint and wire its router into the application routes. Adds app/api/github/github.py implementing GET /github which authenticates via get_api_key, queries the database for record count and column names (information_schema) using get_db_connection_direct, and returns structured meta/data (using make_meta). Adds app/api/github/__init__.py to re-export the router and updates app/api/routes.py to include the github router. Includes basic DB error handling that returns an error meta.
Introduce DB schema and seeding for GitHub ingestion and wire up routes. Adds a POST /api/github/createtable route (app/api/github/sql/create_tables.py) that creates github_accounts, github_repos, github_gists, github_projects, and github_resources tables with indexes. Adds a seed script (app/api/github/sql/seed.py) to insert realistic example rows for accounts, repos, gists, projects, and resources. Updates router initialization (app/api/github/__init__.py) to include the new create-table router. Refactors GET /github (app/api/github/github.py) to return counts and recent rows for each GitHub table, adds a helper to fetch table data, and ensures DB cursors/connections are closed in a finally block. Also updates README to document the create-table route and proposed table design.
@goldlabelapps goldlabelapps self-assigned this Apr 27, 2026
@goldlabelapps goldlabelapps merged commit 70c9c07 into master Apr 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant